Learning control policies of driverless vehicles from UAV video streams in complex urban environments

© 2019 by the authors. The way we drive, and the transport of today are going through radical changes. Intelligent mobility envisions to improve the e°ciency of traditional transportation through advanced digital technologies, such as robotics, artificial intelligence and Internet of Things. Central to the development of intelligent mobility technology is the emergence of connected autonomous vehicles (CAVs) where vehicles are capable of navigating environments autonomously. For this to be achieved, autonomous vehicles must be safe, trusted by passengers, and other drivers. However, it is practically impossible to train autonomous vehicles with all the possible tra°c conditions that they may encounter. The work in this paper presents an alternative solution of using infrastructure to aid CAVs to learn driving policies, specifically for complex junctions, which require local experience and knowledge to handle. The proposal is to learn safe driving policies through data-driven imitation learning of human-driven vehicles at a junction utilizing data captured from surveillance devices about vehicle movements at the junction. The proposed framework is demonstrated by processing video datasets captured from uncrewed aerial vehicles (UAVs) from three intersections around Europe which contain vehicle trajectories. An imitation learning algorithm based on long short-term memory (LSTM) neural network is proposed to learn and predict safe trajectories of vehicles. The proposed framework can be used for many purposes in intelligent mobility, such as augmenting the intelligent control algorithms in driverless vehicles, benchmarking driver behavior for insurance purposes, and for providing insights to city planning

Deep Multi-Critic Network for accelerating Policy Learning in multi-agent environments

Humans live among other humans, not in isolation. Therefore, the ability to learn and behave in multi-agent environments is essential for any autonomous system that intends to interact with people. Due to the presence of multiple simultaneous learners in a multi-agent learning environment, the Markov assumption used for single-agent environments is not tenable, necessitating the development of new Policy Learning algorithms. Recent Actor-Critic algorithms proposed for multi-agent environments, such as Multi-Agent Deep Deterministic Policy Gradients and Counterfactual Multi-Agent Policy Gradients, find a way to use the same mathematical framework as single agent environments by augmenting the Critic with extra information. However, this extra information can slow down the learning process and afflict the Critic with Curse of Dimensionality. To combat this, we propose a novel Deep Neural Network configuration called Deep Multi-Critic Network. This architecture works by taking a weighted sum over the outputs of multiple critic networks of varying complexity and size. The configuration was tested on data collected from a real-world multi-agent environment. The results illustrate that by using Deep Multi-Critic Network, less data is needed to reach the same level of performance as when not using the configuration. This suggests that as the configuration learns faster from less data, then the Critic may be able to learn Q-values faster, accelerating Actor training as well

Use of Machine Learning to Automate the Identification of Basketball Strategies Using Whole Team Player Tracking Data

The use of machine learning to identify and classify offensive and defensive strategies in team sports through spatio-temporal tracking data has received significant interest recently in the literature and the global sport industry. This paper focuses on data-driven defensive strategy learning in basketball. Most research to date on basketball strategy learning has focused on offensive effectiveness and is based on the interaction between the on-ball player and principle on-ball defender, thereby ignoring the contribution of the remaining players. Furthermore, most sports analytical systems that provide play-by-play data is heavily biased towards offensive metrics such as passes, dribbles, and shots. The aim of the current study was to use machine learning to classify the different defensive strategies basketball players adopt when deviating from their initial defensive action. An analytical model was developed to recognise the one-on-one (matched) relationships of the players, which is utilised to automatically identify any change of defensive strategy. A classification model is developed based on a player and ball tracking dataset from National Basketball Association (NBA) game play to classify the adopted defensive strategy against pick-and-roll play. The methodology described is the first to analyse the defensive strategy of all in-game players (both on-ball players and off-ball players). The cross-validation results indicate that the proposed technique for automatic defensive strategy identification can achieve up to 69% accuracy of classification. Machine learning techniques, such as the one adopted here, have the potential to enable a deeper understanding of player decision making and defensive game strategies in basketball and other sports, by leveraging the player and ball tracking data

A machine learning framework for quantifying in-game space-control efficiency in football

Analysis of player tracking and event data in football matches is used by the coaching staff to evaluate team performance and to inform tactical decision-making, whereas using Machine Learning methods to gain useful insights from the data is still an open research question. The objective of our research is to discover the football team's space-control efficiency using a novel Machine Learning approach and evaluate the team performance based on its space-control efficiency. We develop a novel Possession Evaluation Model through deep generative machine learning to predict the football team's space-control capability utilising tracking and event data. The developed model is used to quantify the efficiency of attacking and defending for a given sequence of play. Performance analysis results demonstrate that this novel method of space-control efficiency quantification is objective and precise. The superior performance of the model is attributed to the utilization of deep generative modelling on image datasets and conditioning in the prediction with contextual factors. This study presents a novel approach to football analysis in evaluating team performance and providing tactical insights for the coach to make data-informed adjustments.</p

A multimodal perception-driven self evolving autonomous ground vehicle

Increasingly complex automated driving functions, specifically those associated with Free Space Detection (FSD), are delegated to Convolutional Neural Networks (CNN). If the dataset used to train the network lacks diversity, modality or sufficient quantities, the driver policy that controls the vehicle may induce safety risks. Although most autonomous ground vehicles (AGV) perform well in structured surroundings, the need for human intervention significantly rises when presented with unstructured niche environments. To this end, we developed an AGV for seamless indoor and outdoor navigation to collect realistic multimodal data streams. We demonstrate one application of the AGV when applied to a self-evolving FSD framework that leverages online active machine learning (ML) paradigms and sensor data fusion. In essence, the self-evolving AGV queries image data against a reliable data stream, ultrasound, before fusing the sensor data to improve robustness. We compare the proposed framework to one of the most prominent free space segmentation methods, DeepLabV3+ [1]. DeepLabV3+ [1] is a state-of-the-art semantic segmentation model composed of a CNN and an auto-decoder. In consonance with the results, the proposed framework out preforms DeepLabV3+ [1]. The performance of the proposed framework is attributed to its ability to self-learn free space. This combination of online and active ML removes the need for large datasets typically required by a CNN. Moreover, this technique provides case-specific free space classifications based on information gathered from the scenario at hand.</div

A model selection algorithm for complex CNN systems based on feature-weights relation

In object recognition using machine learning, one model cannot practically be trained to identify all the possible objects it encounters. An ensemble of models may be needed to cater to a broader range of objects. Building a mathematical understanding of the relationship between various objects that share comparable outlined features is envisaged as an effective method of improving the model ensemble through a pre-processing stage, where these objects' features are grouped under a broader classification umbrella. This paper proposes a mechanism to train an ensemble of recognition models coupled with a model selection scheme to scale-up object recognition in a multi-model system. The multiple models are built with a CNN structure, whereas the image features are extracted using a CNN/VGG16 architecture. Based on the models' excitation weights, a neural network model selection algorithm, which decides how close the features of the object are to the trained models for selecting a particular model for object recognition is tested on a multi-model neural network platform. The experiment results show the proposed model selection scheme is highly effective and accurate in selecting an appropriate model for a network of multiple models.</p

LiDAR-based glass detection for improved occupancy grid mapping

Creating an accurate awareness of the environment using laser scanners is a major challenge in robotics and auto industries. LiDAR (light detection and ranging) is a powerful laser scanner that provides a detailed map of the environment. However, efficient and accurate mapping of the environment is yet to be obtained, as most modern environments contain glass, which is invisible to LiDAR. In this paper, a method to effectively detect and localise glass using LiDAR sensors is proposed. This new approach is based on the variation of range measurements between neighbouring point clouds, using a two-step filter. The first filter examines the change in the standard deviation of neighbouring clouds. The second filter uses a change in distance and intensity between neighbouring pules to refine the results from the first filter and estimate the glass profile width before updating the cartesian coordinate and range measurement by the instrument. Test results demonstrate the detection and localisation of glass and the elimination of errors caused by glass in occupancy grid maps. This novel method detects frameless glass from a long range and does not depend on intensity peak with an accuracy of 96.2%

Conversational emotion detection and elicitation: a preliminary study

Emotion recognition in conversation is a challenging task as it requires an understanding of the contextual and linguistic aspects of a conversation. Emotion recognition in speech has been well studied, but in bi-directional or multi-directional conversations, emotions can be very complex, mixed, and embedded in context. To tackle this challenge, we propose a method that combines state-of-the-art RoBERTa model (robustly optimized BERT pretraining approach) with a Bidirectional long short-term memory (BiLSTM) network for contextualized emotion recognition. RoBERTa is a transformer-based language model, which is an advanced version of the well-known BERT. We use RoBERTa features as input to a BiLSTM model that learns to capture contextual dependencies and sequential patterns in the input text. The proposed model is trained and evaluated on a Multimodal EmotionLines Dataset (MELD) to recognize emotions in conversation. The textual modality of the dataset is utilized for the experimental evaluation, with the weighted average F1 score and accuracy used as performance metrics. The experimental results indicate that the incorporation of a pre-trained transformer-based language model with a BiLSTM network significantly enhances the recognition of emotions in contextualized conversational settings.</p

Learning data-driven decision-making policies in multi-agent environments for autonomous systems

Autonomous systems such as Connected Autonomous Vehicles (CAVs), assistive robots are set improve the way we live. Autonomous systems need to be equipped with capabilities to Reinforcement Learning (RL) is a type of machine learning where an agent learns by interacting with its environment through trial and error, which has gained significant interest from research community for its promise to efficiently learn decision making through abstraction of experiences. However, most of the control algorithms used today in current autonomous systems such as driverless vehicle prototypes or mobile robots are controlled through supervised learning methods or manually designed rule-based policies. Additionally, many emerging autonomous systems such as driverless cars, are set in a multi-agent environment, often with partial observability. Learning decision making policies in multi-agent environments is a challenging problem, because the environment is not stationary from the perspective of a learning agent, and hence the Markov properties assumed in single agent RL does not hold. This paper focuses on learning decision-making policies in multi-agent environments, both in cooperative settings with full observability and dynamic environments with partial observability. We present experiments in simple, yet effective, new multi-agent environments to simulate policy learning in scenarios that could be encountered by an autonomous navigating agent such as a CAV. The results illustrate how agents learn to cooperate in order to achieve their objectives successfully. Also, it was shown that in a partially observable setting, an agent was capable of learning to roam around its environment without colliding in the presence of obstacles and other moving agents. Finally, the paper discusses how data-driven multi agent policy learning can be extended to real-world environments by augmenting the intelligence of autonomous vehicles

Understanding dilated mathematical relationship between image features and the convolutional neural network’s learnt parameters

Deep learning, in general, was built on input data transformation and presentation, model training with parameter tuning, and recognition of new observations using the trained model. However, this came with a high computation cost due to the extensive input database and the length of time required in training. Despite the model learning its parameters from the transformed input data, no direct research has been conducted to investigate the mathematical relationship between the transformed information (i.e., features, excitation) and the model’s learnt parameters (i.e., weights). This research aims to explore a mathematical relationship between the input excitations and the weights of a trained convolutional neural network. The objective is to investigate three aspects of this assumed feature-weight relationship: (1) the mathematical relationship between the training input images’ features and the model’s learnt parameters, (2) the mathematical relationship between the images’ features of a separate test dataset and a trained model’s learnt parameters, and (3) the mathematical relationship between the difference of training and testing images’ features and the model’s learnt parameters with a separate test dataset. The paper empirically demonstrated the existence of this mathematical relationship between the test image features and the model’s learnt weights by the ANOVA analysis